Sparse modeling of neural network posterior probabilities for exemplar-based speech recognition

We study automatic speech recognition by direct use of acoustic features (exemplars) without any assumption on the underlying stochastic process. The prior studies exploit spectral exemplars. In this work, we present the use of neural network sub-word posterior probabilities as exemplars. The space of sub-word observations is lowdimensional (e.g. RK×T ) whereas the word transcription requires reconstructing a high-dimensional representation (e.g. RL×T , L K). Given the prior knowledge that for any given utterance, the word representation is highly sparse, we cast the speech recognition problem as sparse reconstruction of word posteriors given the compressed (low-dimensional) acoustic observation. The sub-word units (phones) are denoted by {qk}k=1. Given an input spectral feature xt at time t, a (deep) neural network [1] is used to estimate the posterior probabilities {p(qk|xt)}k=1. The phone posterior probabilities is related to the word posterior probabilities p(wl|xt) through